Method for predicting defects in assembly units

ABSTRACT

One variation of a method for predicting manufacturing defects includes: accessing a set of inspection images of a set of assembly units recorded by an optical inspection station; for each inspection image in the set of inspection images, detecting a set of features in the inspection image and generating a vector representing the set of features in a multi-dimensional feature space; grouping neighboring vectors in the multi-dimensional feature space into a set of vector groups; and, in response to receipt of a first inspection result indicting a defect in a first assembly unit, in the set of assembly units, associated with a first vector in a first vector group, in the set of vector groups, labeling the first vector group with the defect and flagging a second assembly unit associated with a second vector, in the first vector group, as exhibiting characteristics of the defect.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Application claims the benefit of U.S. Provisional Application No.62/485,209, filed on 13 Apr. 2017, which is incorporated in its entiretyby this reference.

TECHNICAL FIELD

This invention relates generally to the field of optical inspection andmore specifically to a new and useful method for predicting defects inassembly units in the field of optical inspection.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a flowchart representation of a method; and

FIG. 2 is a flowchart representation of one variation of the method;

FIG. 3 is a flowchart representation of one variation of the method;

FIG. 4 is a flowchart representation of one variation of the method;

FIG. 5 is a flowchart representation of one variation of the method; and

FIG. 6 is a flowchart representation of one variation of the method.

DESCRIPTION OF THE EMBODIMENTS

The following description of embodiments of the invention is notintended to limit the invention to these embodiments but rather toenable a person skilled in the art to make and use this invention.Variations, configurations, implementations, example implementations,and examples described herein are optional and are not exclusive to thevariations, configurations, implementations, example implementations,and examples they describe. The invention described herein can includeany and all permutations of these variations, configurations,implementations, example implementations, and examples.

1. Methods

As shown in FIGS. 1 and 2, a method S100 includes: accessing a set ofinspection images of a set of assembly units, of a particular assemblytype, recorded by an optical inspection station during production of theset of assembly units in Block S110; for each inspection image in theset of inspection images, detecting a set of features in the inspectionimage in Block S120 and generating a vector, in a set of vectors,representing the set of features in a multi-dimensional feature space inBlock S124; and grouping neighboring vectors, in the set of vectors, inthe multi-dimensional feature space into a set of vector groups in BlockS130.

1.1 Method: Intelligent Recall and Defect Propagation

In one variation shown in FIGS. 1 and 5 in which the method S100 isimplemented to propagate detection of defects in assembly unitscompleted in the past, the method S100 also includes, in response toreceipt of a first inspection result indicating a defect in a firstassembly unit, in the set of assembly units, associated with a firstvector in a first vector group, in the set of vector groups: labelingthe first vector group with the defect in Block S140; and flagging asecond assembly unit associated with a second vector, in the firstvector group, as exhibiting characteristics of the defect in Block S150.

1.2 Method: Defect Source Isolation

In another variation shown in FIG. 2 in which the method S100 isimplemented to predict defect modes (e.g., groups of features thatindicate failure or defects in assembly units), the method S100 alsoincludes: labeling a first vector group, in the set of vector groups,with a defect indicated in inspection results of a first subset ofassembly units, in the set of assembly units, associated with vectors inthe first vector group in Block S140; labeling a second vector group, inthe set of vector groups, with absence of the defect based on inspectionresults of a second subset of assembly units, in the set of assemblyunits, associated with vectors in the second vector group in Block S140;identifying a model set of feature ranges represented in vectors in thefirst vector group and distinct from vectors in the second vector groupin Block S160; and associating the model set of features with the defectin Block S162.

In this variation, the method S100 can be similarly implemented topredict groups of features that indicate absence of a defect in assemblyunits. In this variation, the method S100 can additionally oralternatively include: identifying a model set of features common tovectors in the second vector group and excluded from vectors in thefirst vector group in Block S160; and associating the model set offeatures with proper operation of the particular function for theparticular assembly type in Block S162.

1.3 Method: Real-Time Yield Protection

In another variation shown in FIG. 6 in which the method S100 isimplemented to detect manufacturing drift toward characteristics ofdefective assembly units over time, the method S100 also includes:labeling a first vector group, in the first set of vector groups, with adefect based on a first inspection result indicating the defect in afirst assembly unit, in the first set of assembly units, associated witha first vector in the first vector group in Block S140; accessing asecond sequence of inspection images of a second set of assembly units,of the particular assembly type, recorded by the optical inspectionstation during production of the second set of assembly units over asecond period of time succeeding the first period of time; for eachinspection image in the second sequence of inspection images, detectinga set of features in the inspection image and generating a vector, in asecond set of vectors, representing the set of features in themulti-dimensional feature space; and, in response to the second set ofvectors trending toward the first vector group over the second period oftime, generating a prompt to inspect production preceding the opticalinspection station in Block S170.

In a similar variation in which the method S100 is implemented to detectmanufacturing drift away from characteristics of functional assemblyunits over time, the method S100 also includes: labeling a first vectorgroup, in the first set of vector groups, as functional based on a firstinspection result indicating complete functionality of a first assemblyunit, in the first set of assembly units, associated with a first vectorin the first vector group in Block S140; accessing a second sequence ofinspection images of a second set of assembly units, of the particularassembly type, recorded by the optical inspection station duringproduction of the second set of assembly units over a second period oftime succeeding the first period of time; for each inspection image inthe second sequence of inspection images, detecting a set of features inthe inspection image and generating a vector, in a second set ofvectors, representing the set of features in the multi-dimensionalfeature space; and, in response to the second set of vectors trendingoutwardly from the first vector group over the second period of time,generating a prompt to inspect production preceding the opticalinspection station.

1.4 Method: Real-Time Anomaly Detection

In yet another variation shown in FIG. 3 in which the method S100 isimplemented to detect anomalies in new assembly units during production,the method S100 also includes: accessing a second inspection image of asecond assembly unit, of the particular assembly type, recorded by theoptical inspection station during production of the second assembly unitat a second time succeeding the first period of time; detecting a secondset of features in the second inspection image; generating a secondvector representing the second set of features in the multi-dimensionalfeature space; and, in response to the second vector deviating from theset of vector groups by more than a threshold difference, flagging thesecond assembly unit in Block S150.

2. Applications

Generally, the method S100 can be executed by a local or remote computersystem (hereinafter the “system”) to: aggregate digital photographicinspection images of assembly units recorded during production; torepresent each of these assembly units as a multi-dimensional (e.g., a“n-dimensional”) vector embodying multiple (e.g., “n-number” of)features detected and extracted from a corresponding inspection image;and to group these vectors into groups (or “clusters”) of vectorsexhibiting (relatively) high degrees of similarity in some or alldimensions with a multi-dimensional feature space. In particular, themethod S100 can be executed to automatically identify representativefeatures (e.g., corners, edges, surfaces, surface profiles, geometries,relative positions, relative orientations, etc.) from inspection imagesof many assembly units and to automatically identify groups of similarassembly units—which may exhibit similar aesthetic and functionalsoundness or defects—based on these features but without direct orexplicit knowledge of how these features affect aesthetic and functionalsoundness of these assembly units.

As described below, the system can execute Blocks of the method S100: togenerate asynchronous and real-time predictions of aesthetic and/orfunctional defects of assembly units; to identify anomalous assemblyunits based predominantly or exclusively on visual data of theseassembly units (and limited test and inspection data); to isolatefeatures that are predictive of defects; and to detect unintendedchanges occurring during production of assembly units along an assemblyline well before these unintended changes affect yield.

2.1 Application: Intelligent Recall and Defect Propagation

In one application shown in FIG. 5, the system: accesses a database ofinspection images—of a corpus of assembly units produced in thepast—recorded by an optical inspection station arranged after aparticular assembly step on an assembly line; segments these inspectionimages; passes image segments (e.g., a subset of image segments ofinterest associated with the optical inspection station) from each ofthese inspection images into a convolutional neural network that detectsand extracts features (e.g., thousands of features) from each imagesegment; and then compiles quantitative representations of thesefeatures into one vector for each assembly unit. The system can then:implement structured data analysis techniques (e.g., linear regressionanalysis, cluster analysis, k-means clustering, and/or other statisticalanalysis and machine learning techniques) to group vectors representingassembly units into discrete vector groups (hereinafter “clusters”);write inspection results for a small proportion (e.g., 5%) of theseassembly units (e.g., presence of a defect, complete functionality) toclusters containing vectors corresponding to these inspected assemblyunits; and then predict similar defects (or proper function) in other,uninspected assembly units based on proximity of their correspondingvectors to clusters labeled with defect labels (or with proper functionlabels).

When a newly-identified defect is identified in a particular assemblyunit (e.g., in an assembly unit that was previously sold, later returneddue to a defect, and inspected), the system can: interface with a user(e.g., an engineer, an operator) to update a record of the particularassembly unit to reflect this defect; automatically label a cluster ofvectors containing a particular vector corresponding to the particularassembly unit with this newly-identified defect; identify a set vectorsin or near the same cluster as the particular vector in the featurespace; and transform this set of vectors into a list of sold assemblyunits to selectively recall and/or into a list of assembly units stillonsite to selectively inspect for this newly-identified defect.

The system can therefore process existing inspection images of completedassembly units asynchronously according to the method S100 in order toidentify a subset of these completed assembly units likely to exhibitthis newly-identified defect. More specifically, the system can executeBlocks of the method S100 to automatically “look back” through a corpusof inspection images of previously-completed assembly units for assemblyunits that exhibit features indicative of a newly-identified defect.Such insights generated by the system can then be leveraged by a user(e.g., a design engineer, a quality control engineer, a manufacturingengineer, a manufacturer): to identify and selectively recall assemblyunits likely to be defective (i.e., rather than an entire batch ofassembly units); and/or to guide selective, intelligent inspection andtesting of assembly units that are still onsite in order to quicklyidentify and cull assembly units that are aesthetically or functionallydefective prior to shipment out of the facility.

2.2 Application: Real-Time Defect Detection

As shown in FIG. 4, the system can similarly execute Blocks of themethod S100 substantially in real-time: to compare inspection images ofnew assembly units—recorded by an optical inspection station duringproduction of these new assembly units—to past assembly units of knownoutcomes in order to identify particular new assembly units that exhibitfeatures characteristic of past defective assembly units; and to serveprompts to inspect, discard, or rework these particular assembly unitsin real-time, such as to the optical inspection station or to anoperator nearby.

For example, the system can execute Blocks of the method S100 toselectively flag new assembly units that may be defective based onsimilarity of features detected in inspection images of these newassembly units and features detected in inspection images of past,known-defective assembly units and to issue prompts to a user to addressthese select new units in order to: reduce waste of operator time,assembly line capacity, and components by culling assembly unitsdetermined to be defective even before completion; and improvingefficiency of inspection on the assembly line by isolating particularassembly units that may be defective for inspection.

2.3 Application: Real-Time Anomaly Detection

As shown in FIG. 3, the system can also execute Blocks of the methodS100 in real-time to: receive an inspection image of a new assemblyunit; to extract features from this inspection image; to compare thesefeatures to features extracted from inspection images of other assemblyunits (e.g., of known outcomes); and to flag this new assembly unit asanomalous if this new assembly unit exhibits a feature or a group offeatures that are not present in or that are substantially distinct fromother assembly units. For example, the system can flag the new assemblyunit if a vector representing features extracted from the inspectionimage of the new assembly unit fails to intersect one existing clusterof vectors associated with assembly units of known outcomes or thatfails to fall within a threshold difference (e.g., a threshold distancein Euclidean space) of a nearest existing cluster in themulti-dimensional feature space.

Because the outcome of the new assembly unit (e.g., whether the newassembly unit exhibits any aesthetic or functional defect) is not known,the system can prompt a user (e.g., a technician, an assembly lineoperator, or an engineer) to inspect the new assembly unit for defects,such as in its current assembly state and/or upon completion of the newassembly unit. In the foregoing example, the system can then define anew cluster containing the vector representing this new assembly unit,label this new cluster with an inspection result of the new assemblyunit, and predict similar outcomes for future assembly units thatexhibit similar features. The system can therefore execute Blocks of themethod S100: to detect assembly units that are anomalous within apopulation of assembly units; to issue selective, intelligent prompts toinspect such anomalous assembly units; and to improve a model linkingassembly unit features to assembly unit outcomes based on inspectionresults of such anomalous assembly units.

2.4 Application: Defect Source Isolation

By also identifying a set of features common to a first set of assemblyunits exhibiting a defect but not common to (e.g., excluded from) asecond set of assembly units not exhibiting this defect, the system canfilter a relatively large feature set down to a compressed feature setexhibiting a greater likelihood of producing the defect, as shown inFIG. 2. By scanning inspection images of other assembly units—of thesame or different assembly type—for similarity to this compressedfeature set, the system can detect or predict similar defects in theseother assembly units even if the source of the defect is not immediatelyknown to an engineer.

The system can also present this limited feature set to a user (e.g., anengineer) for manual labeling and identification of specific features inthis set that the user hypothesizes or has confirmed contributed to thedefect. In particular, though the system can extract thousands ormillions of features from an inspection image and process these featuresto identify both assembly units that exhibit similarities to pastassembly units of known outcomes and anomalous assembly units, thislarge number of features may be too large for one or several humans tocomprehend. Therefore, by comparing features extracted from inspectionimages of sound and defective assembly units, the system can: isolate aparticular subset (e.g., dozens) of features exhibiting strongestcorrelation with this defect; highlight regions of inspection imagescontaining these particular features; and present these highlightedinspection images to a user (e.g., through a user portal) forconsideration of the source of this defect.

The system can thus guide a human user to test correlations betweenselect features and a defect, to confirm causation between thesefeatures and the defect, and to label these features accordingly,thereby enabling supervised machine learning pathways via intelligentpresentation of select inspection images—or select regions of interestof these inspection images—predicted by the system to contain featuresindicative of the defect.

2.5 Application: Real-Time Yield Protection

Furthermore, the system can execute Blocks of the method S100 to detecta trend (or “drift”) of features of new assembly units produced on theassembly unit toward features characteristic of past defective assemblyunits and/or away from features characteristic of past non-defectiveassembly units, even before such unintended changes result in a decreasein yield on the assembly line, as shown in FIG. 6. For example,characteristics of assembly units produced on an assembly line may trendor “drift” in a particular direction over time due to tooling wear,fixture wear, decreased oversight (e.g., per unit as production rateincreases), personnel changes, etc. Though assembly units produced earlyin the course of this trend may be non-defective and through this trendmay not initially affect yield at the assembly line, this trend may leadto defective assembly units over a longer period of time (and may bemore difficult to correct) if not timely addressed.

The system can therefore: derive a trend of new vectors—representing anew sequence of assembly units imaged by an optical inspection stationalong the assembly line—toward a cluster of vectors labeled with adefect; and/or derive a trend of new vectors away from (e.g., outwardlyfrom) a cluster of vectors labeled as non-defective. If a strength orrate of this trend exceeds a threshold, the system can then promptinspection of a segment of the assembly preceding the optical inspectionstation for a possible source of this trend, thereby enabling anengineer or technician to quickly verify and address this trend, such asby reinforcing protocols for operators on the assembly line or replacingtooling upstream of the assembly line (e.g., an injection mold).

The system can therefore execute Blocks of the method S100 to detectunintended changes within production of assembly units and to promptinspection and correction of these changes prior to a substantivedecrease in yield on the assembly line based on features extracted frominspection images of these assembly units recorded by optical inspectionstations arranged along the assembly line.

3. System

Blocks of the method S100 can be executed by a system, such as: locallyon a system integrated into an optical inspection station (as describedbelow) at which inspection images of assembly units are recorded;locally at a system near an assembly line populated with opticalinspection stations; or remotely by a remote server connected to opticalinspection stations via a computer network (e.g., the Internet), etc.The system can also interface with a user portal—such as accessiblethrough a web browser or native application executing on a laptopcomputer or smartphone—to serve prompts and notifications to a user(e.g., an engineer or assembly line worker) and to receive feature andcluster labels entered manually by a user.

Furthermore, the method S100 is described below as executed by thesystem to detect anomalies and/or defects in assembly units containingtwo or more discrete components. However, the method S100 can besimilarly implemented by a system to detect anomalies and/or defects insingular parts (e.g., molded, cast, stamped, or machined parts) based oninspection images of these singular parts.

All or select Blocks of the method S100 can additionally oralternatively be executed locally at the assembly line, such as by anoptical inspection station once the optical inspection station recordsan inspection image of a new assembly unit inserted into the opticalinspection station for automated imaging and optical inspection.

4. Optical Inspection Station and Inspection Images

Block S110 of the method S100 recites accessing a set of inspectionimages of a set of assembly units, of a particular assembly type,recorded by an optical inspection station during production of the setof assembly units. Generally, in Block S110, the system accessesinspection images recorded by an optical inspection station duringassembly of assembly units. For example, the system can retrieveinspection images recorded by an optical inspection station, uploadedfrom the optical inspection station to a file system (e.g., a database)via a computer network, and stored in a database. The system canadditionally or alternatively retrieve inspection images directly fromthe optical inspection station, such as in real-time when an inspectionimage of an assembly unit is recorded by the optical inspection station.

As described in U.S. patent application Ser. No. 15/653,040, filed on 18Jul. 2017, which is included in its entirety by this reference, anoptical inspection station can include: an imaging platform thatreceives a part or assembly; a visible light camera (e.g., a RGB CMOS,or black and white CCD camera) that captures inspection images (e.g.,digital photographic color images) of units placed on the imagingplatform; and a data bus that offloads inspection images, such as to alocal or remote database. An optical inspection station can additionallyor alternatively include multiple visible light cameras, one or moreinfrared cameras, a laser depth sensor, etc.

In one implementation, an optical inspection station also includes adepth camera, such as an infrared depth camera, configured to outputdepth images. In this implementation, the optical inspection station cantrigger both the visible light camera and the depth camera to capture acolor image and a depth image, respectively, of each unit placed on theimaging platform. Alternatively, the optical inspection station caninclude optical fiducials arranged on and/or near the imaging platform.In this implementation, the optical inspection station (or a local orremote computer system interfacing with the remote database) canimplement machine vision techniques to identify these fiducials in acolor image captured by the visible light camera and to transform sizes,geometries (e.g., distortions from known geometries), and/or positionsof these fiducials within the color image into a depth map, into athree-dimensional color image, or into a three-dimensional measurementspace (described below) for the color image, such as by passing thecolor image into a neural network.

Upon receipt or retrieval of an inspection image, the system can“dewarp,” “flatten,” or otherwise preprocess the inspection image inBlock S110 in preparation for detecting and extracting features from theinspection image in Blocks S120 and S122, as described in U.S. patentapplication Ser. No. 15/407,158, filed on 16 Jan. 2017, which isincorporated in its entirety by this reference. The system can also:implement computer vision techniques (e.g., object recognition, edgedetection, etc.) to identify a perimeter or boundary of the assemblyunit in the inspection image; and then crop the inspection image aroundthe assembly unit such that only features corresponding to the assemblyunit are extracted from the inspection image and processed in BlocksS120, S122, etc. of the method S100.

The system can thus aggregate a set of (e.g., 100, 1 k, or 100 k)inspection images (e.g., digital color photographic image) recorded overa period of operation of an assembly line in Block S110, wherein eachinspection image records visual characteristics of a unique assemblyunit at a particular production stage. However, the system can accessinspection images of any other type and in any other way in Block S110.

5. Feature Extraction and Vector Generation

The method S100 also includes Blocks S120, S122, and S124, which recite,in each inspection image in the set of inspection images: detecting aset of features in the inspection image; extracting the set of featuresfrom the inspection image; and generating a multi-dimensional vector, ina set of vectors, representing the set of features, respectively.Generally, the system identifies multiple (e.g., “n,” or “many”)features representative of an assembly unit shown in an inspection imagein Block S120, characterizes these features in Block S122, andaggregates these features into a multi-dimensional (e.g.,“n-dimensional”) vector or “fingerprint” uniquely representing thisassembly unit—such lengths, angles, relative positions, relativeorientations, geometries, profiles, etc. of these features—in BlockS124.

In one implementation, the system can implement a feature classifierthat defines types of single-order features (e.g., corners, edges,areas, gradients, etc.), second-order features constructed from multiplesingle-order features (edge orientation and gradient magnitude of anedge, polarity and strength of a blob), relative positions andorientations of multiple features, and/or prioritization for detectingand extracting these features from an inspection image, etc. The systemcan then apply this feature classifier to the full height and width of aregion of the inspection image representing the assembly unit. Forexample, the system can implement: low-level computer vision techniques(e.g., edge detection, ridge detection); curvature-based computer visiontechniques (e.g., changing intensity, autocorrelation); and/orshape-based computer vision techniques (e.g., thresholding, blobextraction, template matching); etc. according to the feature classifierto detect n-number of highest-priority features representing theassembly unit in the inspection image in Block S120.

The system can then extract a local image patch around these features inBlock S122, such as in the form of a multi-dimensional (e.g.,n-dimensional) feature vector (hereinafter a “vector”) representingmultiple (e.g., thousands, millions) features extracted from theinspection image. The vector can thus define a “fingerprint” thatuniquely represents the assembly unit captured in the inspection image.

The system can repeat this process for other inspection images—such asby processing these inspection images in a batch or by processing newinspection images individually upon receipt from an optical inspectionstation—to generate vectors uniquely representing corresponding assemblyunits.

5.1 Zoom+Region of Interest

In one implementation shown in FIGS. 3 and 4, the system limits featuredetection and extraction in an inspection image to a region of interestin the inspection image defined or bound manually by a user. Inparticular, the system can be configured to extract a preset number of(e.g., two thousand) features from an inspection image; by reducing ascan area of the inspection image, the system can detect and extract ahigher density of features from this reduced scan area than from thefull assembly unit area of the inspection image.

Generally, given a fixed vector length, the system can: extract a lowerdensity of features from a full height and width of a region of aninspection image corresponding to an assembly unit; and extract agreater density of features from a subregion of the inspection imagecorresponding to the assembly unit. The system can implement the formertechnique by default in order to characterize an assembly unitgenerally. However, if a user (e.g., an engineer) is testing anhypothesis for a source of a defect within a particular subregion of anassembly type, has sourced a root cause of a defect to a particularsubregion of an assembly type, or has identified a defect in aparticular subregion of an assembly type, the user can select thisparticular subregion of the assembly unit to bound detection andextraction of the fixed number of features from the inspection image;the system can then extract a higher density of features from thisbounded region of inspection images of assembly units, which may enablethe system to detect and characterize smaller features that maycontribute (significantly) to a defect in the assembly unit but whichmay have otherwise been assigned lower priorities by the featureclassifier.

In one implementation, the system can implement methods and techniquesdescribed in U.S. patent application Ser. No. 15/407,158 to: serve aninspection image to a user through a user portal executing on acomputing device (e.g., a mobile computing device, a tablet, or adesktop computer); enable the user to zoom into (or “expand”) a regionof the inspection image; and then define a boundary (e.g., a rectangular“zoom window”) around this region of the inspection image. In thisimplementation, the user can thus define a “region of interest” of theinspection image depicting a region of an assembly unit: that changedfrom a preceding production stage to a depicted production stage of theassembly unit; in which the user anticipates a manufacturing defectoccurring; and/or that the user desires to track for manufacturingchanges. In particular, the system can host the user portal and serve aninspection image of an assembly unit to the user portal for selection ofa region of interest in this inspection image, which corresponds to aregion of interest in the assembly unit.

In this implementation, the system can then store this region ofinterest and associate this region of interest with the opticalinspection station that recorded the inspection image. (Alternatively,the system can associate this region of interest with the particularproduction stage and orientation of the assembly unit relative to anoptical inspection station when the inspection image was recorded.) Forexample, the system can: automatically detect a reference feature in theinspection image or prompt the user to select the reference feature inthe inspection image rendered in the user portal, such as a top-leftcorner of the assembly unit or a corner of a component on the assemblyunit visible in the inspection image; define a lateral position,longitudinal position, orientation, width, and height of the region ofinterest relative to this reference feature in the inspection image; andstore the lateral position, longitudinal position, orientation, width,and height in association with the optical inspection station. Thesystem can then selectively detect features in this region of interestof the inspection image in Block S120 and generate a vector thatrepresents this region of the assembly unit in Block S124.(Alternatively, the system can append an existing vector assigned tothis assembly unit with representations of these features and repeatthis process at other production stages of the assembly unit to generatea signal vector assigned to this assembly unit and containingrepresentations of feature sets extracted from regions of interest of aseries of inspection images of the assembly unit recorded duringproduction of the assembly unit over time.)

With the region of interest for the optical inspection station (or forassembly type and production stage of the assembly unit) thus defined,the system can project the region of interest onto inspection images ofother assembly units recorded by the optical inspection station (or ofthe same assembly type and at the same production stage) based onsimilar reference features detected in these inspection images. Forexample, the system can: implement object detection, object recognition,or template matching techniques to identify the reference featuredescribed above in inspection images of other assembly units at the sameproduction stage; and orient the region of interest on these otherinspection images automatically based on lateral position, longitudinalposition, orientation, width, and height values of the region ofinterest assigned to this optical inspection station. Alternatively, thesystem can define the region of interest relative to a global referenceon the inspection image, such as a top-left corner of the inspectionimage. The system can then: detect feature sets in these bounded regionsof interest in these other inspection images in Block S120; compilethese feature sets into vectors assigned to corresponding assembly unitsin Block S124; and then compare these vectors—representing similarregions of interest in inspection images of many assembly units—toidentify clusters of vectors representing similar assembly units inBlock S130.

In another implementation, if a defect in a first assembly unit isidentified through testing or inspection, a user may access the userportal to label a first inspection image of the first assembly unit asdefective and to select a region of interest (or “zoom window,” or“boundary”) around a location of this defect depicted in a firstinspection image of the first assembly unit. The system can thus linkthis region of interest with both the optical inspection station (or thewith this production stage and assembly unit orientation) and theparticular defect. The system can then: project this region of interestonto inspection images of assembly units recorded by this opticalinspection station; detect feature sets in these bounded regions ofinterest in these inspection images in Block S120; compile these featuresets into vectors assigned to corresponding assembly units in BlockS124; identify a first cluster of vectors—including a first vectorassociated with the first assembly unit—representing assembly unitslikely to exhibit the particular defect; and identify another cluster ofvectors representing assembly units less likely or unlikely to exhibitthe particular defect.

Therefore, the system can: enable the user to manually select an area ofinterest on an assembly unit shown within an inspection image in orderto leverage the user's understanding of defects, problematic areas, orkey functions of the assembly unit; project this region of interest ontomany inspection images of assembly units in production on the assemblyline; extract features at greater resolution, smaller features, and/orfeatures assigned lower weight or lower priority by the featureclassifier from like regions of interest in the inspection images; andthen generate vectors (or other data containers) that represent thesefeatures extracted from comparable regions of interest across theseinspection images. In particular, the system can generate (or append)vectors based on features extracted from focused regions of interest ininspection images of like assembly units, thereby necessitating lesstime and computing power to generate higher-resolution representationsof key regions of assembly units for comparison in Block S130, which mayimprove accuracy and resolution of subsequent defect, anomaly, andproduction drift detection described below.

5.2 Image Segmentation

In one variation shown in FIG. 3, when processing an inspection image ofan assembly unit, the system: divides the inspection image into a set ofimage segments (e.g., 224-pixel square “crop areas”); and selects asubset of image segments—in this set of image segments—that correspondsto an assembly unit region of interest associated with the opticalinspection station—in a set of optical inspection stations arrangedalong an assembly line—that recorded the inspection image. The systemcan then: extract a subset of features from each image segment in thissubset of image segments; compile these subsets of features—extractedfrom the subset of image segments intersecting the assembly unit regionof interest in the inspection image—into one set of targeted features inthe inspection image in Block S122; and generate a vector representingthis set of targeted features in the multi-dimensional feature space inBlock S124.

Therefore, the system can divide the inspection image into imagesegments and then select a subset of these image segments thatintersects a region of interest that was previously specified by a userfor the optical inspection station that recorded the inspection image orthat was previously assigned to assembly units of the same type and atthe same production stage. The system can then extract features fromeach of these select image segments and compile these features into onevector that represents a region of interest on the assembly unit of thistime and at this production stage.

5.3 Vectors and Production Stage

In the foregoing implementations, the system can thus: generate onevector per assembly unit imaged at one optical inspection station alongthe assembly line; and then compare vectors corresponding to thisparticular assembly type, production stage, and assembly orientation todetect or predict defects in assembly units—asynchronously or inreal-time—occurring during production steps preceding this opticalinspection station, as described below.

Alternatively, the system can implement the foregoing methods andtechniques: to assign one or more regions of interest to each opticalinspection station along the assembly line; to access a sequence ofinspection images of one assembly unit recorded by these opticalinspection stations during assembly of the assembly unit; and to detectand extract features from these regions of interest in these inspectionimages of the assembly unit. The system can then compile thesefeatures—extracted from a sequence of inspection images of the singularassembly unit recorded by multiple optical inspection stations duringproduction of the assembly unit over time—into one vector thatrepresents characteristics of the assembly unit throughout itsproduction. The system can repeat this process to generate comparablevectors for other assembly units and to then compare these vectors todetect or predict defects in these assembly units occurring at variousmanufacturing steps along the assembly line, as described below.

5.4 Region of Interest Labels

In the foregoing implementations, the system can also enable a user tolabel regions of interest with searchable textual labels or “tags”(e.g., keywords), such as “battery”, “speaker”, “antenna”, “fastener”,“touchscreen,” or “ribbon cable.”

For example, when a defective assembly unit is later identified and adefect mode determined (e.g., “battery failure”, “speaker failure”,“antenna failure”, “fastener missing”, “touchscreen failure,” or “ribboncable break”), the user can: search tags of crop areas ofinterest—assigned to optical inspection stations along the assemblyline—for a subset of crop areas of interest that may contain featuresrelated to this failure; select a particular crop area from this subsetthat the user hypothesizes contains a source of the defect in thedefective assembly unit; and select this particular crop area in aninspection image of the defective assembly unit. Once the system loadsthis particular crop area into the user portal for rendering for theuser, the user can: confirm that the particular crop area generallycontains the source of the defect; or select (or “highlight”) a regionof interest within the particular crop area of interest that the user isconfident contains the source of the defect. The system can then: labelthis crop area of interest or region of interest with this defect;extract features from this crop area of interest or region of interest;compile these features into a representative vector in multi-dimensionalfeature space; repeat this process to generate vectors representingfeatures extracted from the same crop area of interest or region ofinterest of inspection images of other assembly units; and predict thesame defect mode in other assembly units based on proximity ofcorresponding vectors to the representative vector, as described below.(In particular, the system can estimate probability of the same defectin these other complete assembly units based on proximity ofcorresponding vectors to the representative vector in themulti-dimensional feature space.)

The system can implement similar methods and techniques: to generate anew vector from features extracted from the same crop area of interestor region of interest in a new inspection image of a new assembly unitrecently recorded by the optical inspection station; and to estimateprobability of the same defect in the new assembly unit—substantially inreal-time—based on proximity of the new vector to the representativevector in the multi-dimensional feature space, as described below.

6. Vector Groups

Block S130 of the method S100 recites grouping vectors, in the set ofvectors, into a set of discrete clusters, as shown in FIGS. 1, 3, and 4.Generally, in Block S130, the system implements structured data analysistechniques (e.g., linear regression analysis, cluster analysis, k-meansclustering, and/or other statistical analysis and machine learningtechniques) to partition the set of vectors—each uniquely representingmultiple features of one assembly unit—into multiple groups or“clusters” of vectors representing similar combinations of featuresand/or similar feature ranges in one or more dimensions in themulti-dimensional feature space.

7. Initial Labeling and Outcome Propagation

One variation of the method S100 includes Block S140, which reciteslabeling a first cluster, in the set of clusters, with a particulardefect indicated in inspection results of a first subset of assemblyunits, in the set of assembly units, associated with vectors in thefirst cluster. Generally, in Block S140, the system links test and/orinspection data of a particular assembly unit to a vector representingthis assembly unit and then associates these test and/or inspection datawith a particular cluster containing the particular vector.

In one implementation, the system automatically accesses test data,inspection results, and/or other observations entered manually by a userand linked to serial numbers of assembly units, such as by retrievingthese data from a database. The system can then extract various defectlabels from these data, such as: poor battery life; speaker malfunctionor audio distortion; poor wireless signal quality; button malfunction;overheating; total system failure; sensor malfunction or noisy sensorsignal; unresponsive touchscreen; distorted display; scratches; dents;and/or rough or uneven surface finish; etc. The system can similarlyextract soundness labels from these data, such as confirmation thatdefined functional and/or aesthetic requirements have been met. Thesystem can also link these defect and/or soundness labels to assemblyunit serial numbers specified in these test, inspection, and/or otherdata and then write these labels to clusters containing vectorsrepresenting these serial numbers in Block S140.

The system can additionally or alternatively prompt a user—via the userportal—to manually tag vectors and/or clusters with defect labels. Forexample, the system can select a first cluster (e.g., containing no orfewer than a minimum number of labels), identify a particular vectorrepresentative of the cluster (e.g., nearest a centroid of the cluster),retrieve a particular inspection image from which the particular vectororiginated, and then prompt the user to select a preexisting label or toenter a new label describing soundness or a defect of the particularassembly unit shown in the particular inspection image. In this example,the system can enable the user to select or enter a label indicatingwhether a scratch is visible in the particular inspection image, whetherthe particular inspection image shows that a specified component ismissing, or whether the assembly unit shown in the particular inspectionimage is known to be defective, etc. The system can then map this labelof the particular assembly unit onto the first cluster.

In the foregoing implementation, if the system defines multiple clustersin Block S130, the system can: select a subset of (e.g., one, five)vectors representative of each cluster; prompt the user to label theinspection image corresponding to these representative vectors withaesthetic and/or functional defect and soundness tags until each clustercontains a sufficient number of labels to predict defects in otherassembly units represented in these clusters.

The system can also prompt the user to manually tag specific featureswith soundness or defect labels. For example, the system can: prompt theuser to select one or a set of features (e.g., one edge, one surface orarea, two edges, an edge and a corner, etc.) within the inspectionimage; prompt the user to label the selected feature(s) asrepresentative of soundness or of a defect; and tag the selectedfeature(s) (e.g., presence of a feature, a length of a feature, distanceor angle between two features, an area of a feature, a geometry orprofile of a feature, etc.) with a defect label entered by the user.

In one example, the system: selects a first vector associated with afirst assembly unit and representative of a first cluster of vectorsassociated with a negative outcome (e.g., presence of a particulardefect); selects a second vector associated with a second assembly unitand representative of a second cluster of vectors associated with apositive outcome (e.g., absence of the particular defect); detects acommon reference feature in a first inspection image of the firstassembly unit and a second inspection image of the second assembly unitat the same product stage; and virtually aligns the first and secondinspection images (or regions of interest in the first and secondinspection images) according to this common reference feature, such asdescribed above and in U.S. patent application Ser. No. 15/653,040. Thesystem can then serve first and second inspection images to a userportal such as rendered within a web browser or native applicationexecuting on a smartphone, tablet, or desktop computer. The user portalcan then toggle between rendering the first inspection image andrendering the second inspection image—virtually aligned by the commonreference feature—such as at a rate of 1 Hz or when triggered by a user,which may enable the user to quickly discern differences between likeregions of interest in the first and second assembly unit and which mayenable the user to isolate a source of a defect present in the firstassembly unit but not in the second assembly unit. In this example, theuser portal can also animate a transition from rendering the firstinspection image to rendering the second inspection image—againvirtually aligned in rotation and orientation by the common referencefeature—which may further enable the user to visually detect differencesbetween the first and second assembly units. The user can then selectand label key features in the first inspection image that the user knowsor hypothesizes contributed to a functional defect or which represent anaesthetic defect, and the system can leverage such feedback from theuser to refine a model of feature sets predicting positive or negativeassembly unit outcomes, as described below.

By thus labeling a cluster of vectors with test and/or inspection dataof assembly units represented by vectors contained in the cluster, thesystem can: infer soundness, aesthetic defects, and/or functionaldefects of other assembly units represented by vectors in this clusterbased on these limited test and/or inspection data; and/or isolatefeatures indicative of such aesthetic and/or functional defects, such asby comparing features represented in different clusters. For example,the system can group a population of vectors in multiple clusters,wherein each cluster containing tens or hundreds of vectors and ones ortens of vectors labels with outcomes (e.g., presence and types ofdefects) of their corresponding assembly units. In this example, thesystem, can thus: predict presence of this particular defect in otherassembly units associated with the other unlabeled vectors in the firstcluster; and serve a prompt to a user (e.g., a manufacturing engineerassociated with the assembly line, a technician on the assembly line) toinspect these other assembly units for the particular defect, such as inreal-time as these other assembly units are assembled or asynchronouslyby retrieving these assembly units from a stock room. Similarly, thesystem can: define a second cluster containing hundreds of vectors,including tens of vectors associated with assembly units confirmed toexclude the particular defect and labeled as non-defective; predictabsence of the defect in assembly units associated with unlabeledvectors in the second cluster; and generate a list of serial numbers ofthese assembly units to exclude from inspection for this particulardefect.

8. Exploration and Problem Solving: Upstream Defect Detection

As shown in FIGS. 1 and 5, one variation of the method S100 includes:generating a prompt to inspect assembly units associated with vectors inthe first cluster for the particular defect in Block S150. Generally, inBlock S150, the system can propagate detection of an aesthetic orfunctional defect across a set of units responsive to detection of thisdefect in one or a small number of assembly units.

In one implementation, if a new functional or aesthetic defect is firstidentified in a particular assembly unit after multiple (e.g., 100 or100 k) like assembly units are produced, the system can: label a firstcluster—containing a particular vector representative of the particularassembly unit—with the defect in Block S130; and flag a set of serialnumbers corresponding to each other vector in the first cluster foradditional testing or inspection related to the newly-identified defect.For example, the system can generate a list of serial numbers to testfor this defect before shipping this batch of assembly units. In anotherexample, if these serial numbers have been marked as shipped ordelivered, the system can generate a list of serial numbers toselectively recall due to possibility of this defect (i.e., rather thanall shipped assembly units).

In another example, the system receives a query from a user (e.g., anengineer) for a first inspection image for a first assembly unit inwhich a newly-identified defect was recently detected, such as in theform of a search query including a serial number of the first assemblyunit entered through a user portal executing on the user's computingdevice. In this example, the first assembly unit may have beencompleted, vended (e.g., to a customer or distributor), and laterreturned to the manufacturer as defective. In this example, the systemcan then: serve a series of inspection images recorded throughoutproduction of the first assembly unit to the user portal; receive anindication of a particular defect in a first inspection image—in thisseries of inspection images—of the first assembly unit at a particularproduction stage; receive manual selection of a region of interest inthe first inspection image via the user portal, wherein the region ofinterest in the first inspection image is predicted by the user todepict a source of the newly-identified defect; and store this region ofinterest with a label for this newly-identified defect. The system canthen scan a population of inspection images of other completed assemblyunits and/or assembly units currently in-process for the same defect. Inparticular, the system can access inspection images of other assemblyunits at the same production stage. For each of these inspection images,the system can then: project the region of interest from the firstinspection image onto an inspection image of another assembly unit;extract a set of features from the region of interest in the inspectionimage; and compile these features into a vector representing this regionof interest in this other assembly unit. Finally, the system can:regroup vectors—in the population of vectors thus generated fromfeatures extracted from this region of interest across many inspectionimages of these many assembly units—based on proximity in themulti-dimensional feature space; and predict presence of thenewly-identified defect in a subset of assembly units associated with asubset of these vectors that fall in (or substantially near) a vectorgroup containing a first vector that represents this region of interestin the first assembly unit. The system can then automatically: generatea first list of serial numbers that correspond to assembly units in thissubset of assembly units and that were previously vended (e.g., deliveryto a customer or distributor); generate a second list of serial numbersthat correspond to assembly units in this subset of assembly units butthat have not yet been vended (e.g., that are still onsite); and/orgenerate a third list of serial numbers that correspond to assemblyunits in this subset of assembly units and that are still in the processof assembly.

The system can execute this process substantially in real-time—followingreceipt of indication of a newly-identified defect in one assemblyunit—to scan and identify other assembly units that may exhibit the samedefect (exclusively) from historical optical data recorded duringproduction of these assembly units. For example, the user or themanufacturer can then: leverage the first list of serial numbers toissue selective, targeted recall of only these assembly units predictedto be defective, such as before this defect substantively affectscustomers of these assembly units; leverage the second list of serialnumbers to selectively retrieve and inspect these assembly units forthis defect prior to delivery to a customer or distributor; and/orleverage the third list of serial numbers to discard these assemblyunits or flag these assembly units for rework.

The system can therefore predict defects in certain past assembly unitsby identifying vectors—representing these assembly units—that aresufficiently near a particular vector representing a defective assemblyunit across multiple (e.g., “n”) dimensions.

8.2 Selective Defect Check

In this variation, the system can also: select a second assembly unitassociated with a second vector proximal the first vector and currentlyonsite or otherwise accessible for inspection; prompt a user to inspectthe second assembly unit and to indicate whether a newly-identifiedfunctional defect is present; and then regroup clusters of vectorsaccording to the user's feedback in order to refine or confirm a largerlist of assembly units predicted to exhibit this same functional defect.

Similarly, if the newly-identified defect is aesthetic, the system canretrieve an inspection image of a second assembly unit predicted toexhibit the same aesthetic defect (e.g., nearest the first vectorrepresenting the first assembly unit confirmed to exhibit this aestheticdefect), serve this inspection image to the user, prompt the user toconfirm presence of the aesthetic defect, and refine or confirm a largerlist of assembly units predicted to exhibit this same aestheticdefect—even if the second assembly unit is not immediately available forphysical inspection by the user.

The system can similarly: identify select assembly units represented byvectors further from the first vector in the multi-dimensional featurespace; prompt the user to physically or visually inspect these selectassembly units for the newly-identified defect; and refine a boundary(or “manifold”) around a cluster of vectors likely to exhibit thisnewly-identified defect and separate from other vectors unlikely or lesslikely to exhibit this defect based on inspection results provided bythe user. For example, the system can: label a first cluster of vectorscontaining the first vector representing the first assembly unit knownto exhibit the newly-identified defect; label a second cluster ofvectors containing a second vector representing a second assembly unitknown to exclude the newly-identified defect based on past inspectionresults; and generate a prompt to inspect a third assemblyunit—associated with a third vector that falls between the first clusterand the second cluster in the multi-dimensional feature space—for thenewly-identified defect. Given inspection results provided by the userfor the third assembly unit, the system can expand one of the first andsecond clusters to include the third vector and repeat this process toincrease the system's confidence in a boundary between the first clusterassociated with the defect and the second cluster associated withabsence of the defect. For example, upon receiving confirmation ofabsence of the defect in the third assembly unit responsive to theprompt to inspect the third assembly unit, the system can: calculate arevised manifold containing the first vector, excluding the thirdvector, and extending between the first vector and the second vector (orotherwise delineating volumes occupied by groups or clusters of vectorsin the multi-dimensional feature space); and generating a fourth promptto inspect a fourth assembly unit associated with a fourth vector,contained within the revised manifold of the first cluster, for thedefect.

Therefore, by selecting key assembly units—represented by vectors thatfall between a first cluster containing a first vector labeled asdefective and a second cluster containing a second vector labeled asnon-defective—for further inspection and prompting a user to indicateoutcomes of these key assembly units, the system can quickly define aperimeter or boundary between defective units and non-defective units(at least for a particular defect) in the multi-dimensional featurespace based on a minimum of human-generated inspection information.

8.3 Defect Probability and Inspection Rank

In this variation (and other variations described herein), the systemcan also calculate probability of a defect in a second assembly unitbased on proximity of a second vector—representing the second assemblyunit of unknown outcome—to a first vector representing a first assemblyunit known to exhibit this defect, as shown in FIG. 4. For example, uponreceipt of indication of a newly-identified defect in a first assemblyunit represented by a first vector, as described below, the system can:calculate distances of other vectors—representing similar regions ofinterest in assembly units of the same type and at the same productionstage—to the first vector in the multi-dimensional feature space; andcalculate probabilities that the defect is present in each of theseassembly units as a function of (e.g., inversely proportional to) thesedistances. The system can then prompt a user to inspect assembly unitsaccording to probability of this defect. For example, the system canserve a list of serial numbers of assembly units in order of probabilityof exhibiting the defect, starting with a second assembly unitassociated with a greatest probability of exhibiting the defect (e.g.,represented by a second vector nearest the first vector) down to athreshold probability (e.g., 20%) of exhibiting the defect. The user canthen inspect these assembly units in the order indicated in this list,such as until an assembly unit that does not exhibit the defect isreached.

9. Real-Time Single-Unit Yield Protection

As shown in FIG. 4, the system can similarly implement Blocks S140 andS150 substantially in real-time to predict defects in assembly unitsduring production. In one implementation, the system: receives a newinspection image of a new assembly unit recently recorded at an opticalinspection station during production of the new assembly unit; detects anew set of features in the new inspection image; extracts the new set offeatures from the new inspection image; generates a newmulti-dimensional vector representing the new set of features; and thenflags the new assembly unit if the new vector representing the newassembly unit intersects a particular cluster labeled with a defect orcontaining a vector tagged with a defect. For example, after generatingthe new vector, the system can locate the new vector in the particularcluster by implementing structured data analysis techniques, such as ak-nearest neighbor classifier (e.g., where k=1). The system can thenprompt a user (e.g., an assembly line operator or an engineer near theassembly line) to reject or correct the new assembly unit, such as bysending a notification to the optical inspection station or to a mobilecomputing device associated with the user before the new assembly unitleaves the optical inspection station. The system can also send acommand to a robotic system proximal the optical inspection station todiscard the assembly or to place the new assembly in a rework bin beforethe new assembly unit moves to a next stage of assembly.

For example, if the defect associated with the particular cluster—andnow with the new assembly unit—is tagged as catastrophic, the system canserve a prompt to the user to discard the new assembly unit. However, ifthe defect associated with the particular cluster is tagged ascorrectable, the system can serve a prompt to the user to correct thenew assembly unit and a predefined instruction for rectifying thedefect. For example, if the predicted defect in the new assembly unit isknown to be correctable, the system can: flag this new assembly unit forrework; retrieve the new inspection image of the assembly unit;highlight a feature representative of the predicted defect or a regionof interest predicted to contain the defect in the new inspection image(e.g., based on defective feature labels in inspection images of otherassembly units represented in the particular cluster); and then servethis inspection image with a textual instruction to review thehighlighted region and to consider the new assembly unit for rework to alocal computing device. In this example, the system can serve thisinspection image and instruction to the optical inspectionstation—currently housing the new assembly unit—for immediate renderingfor a user (e.g., a technician or operator nearby) or serve thisinspection image and instruction to a mobile computing device (e.g., asmartphone or tablet) linked to the user for immediate (e.g., real-time)presentation.

In the foregoing examples, the system can additionally or alternativelyissue an alarm to reject or correct the new assembly unit directlythrough the optical inspection station, such as triggering an audible orvisual alarm to set aside the new assembly unit for further testing orinspection before resuming assembly.

Therefore, the system can implement methods and techniques describedabove to: extract features in a region of interest—defined for anassembly type, orientation, and production stage—in a new inspectionimage of a new assembly unit; to compile the features into a new vectorrepresenting the region of interest on the new assembly unit; andcompare these new vectors to vectors representing like regions ofinterest of past assembly units of known outcomes and/or to clusters ofvectors associated with certain known defects in order to calculate aprobability that the new assembly unit exhibits one or more knowndefects. The system can then selectively prompt various actions relatedto the new assembly unit accordingly. For example, the system canreference a set of rules, such as defined by a user, to: flag the newassembly unit to be discarded if the probability of a terminal defect inthe new assembly unit exceeds a corresponding threshold (e.g., 60%);flag the new assembly unit for rework if the probability of a reworkabledefect in the new assembly unit exceeds a corresponding threshold (e.g.,40%); flag the new assembly unit for further manual inspection upon itscompletion if the probability of at least one defect in the new assemblyunit at this production stage exceeds a corresponding threshold (e.g.,20%); and pass the new assembly unit as likely to be non-defective ifthe probability of any known defect in the new assembly unit at thisproduction stage is less than a corresponding threshold (e.g., 20%).

10. Anomaly Detection

In a similar variation shown in FIG. 3, upon receipt of a new inspectionimage of a new assembly unit, the system can generate a new vectorrepresenting the new assembly unit and then attempt to match the newvector to an existing cluster of vectors representing past assemblyunits. For example, the system can implement a k-nearest neighborclassifier to group the new vector with a nearest existing cluster, asdescribed above. However, if a distance from the vector to the centroidof the nearest cluster exceeds a threshold distance (e.g., two standarddeviations from an average vector-to-centroid distance for thiscluster), the system can flag the new assembly unit as anomalous. Forexample, the system can serve a prompt to the user to inspect the newassembly unit or to set the new assembly unit aside for testing beforeresuming its assembly, as described above.

Additionally or alternatively, if a distance from the vector to thecentroid of the nearest cluster exceeds a threshold distance (e.g., twostandard deviations from an average vector-to-centroid distance for thiscluster), the system can index the number of k clusters by “1” andrepeat the structured data analysis process described above torecalculate “k+1” clusters to convergence. If the new vector representsthe sole vector in a new cluster, the system can again flag the newassembly unit as anomalous. However, if the new cluster contains boththe new vector and one or more other vectors, the system can flag boththe new assembly unit and assembly units associated with the othervectors in the new cluster for further testing and inspection.

In this variation, when an anomalous assembly unit is thus identified,the system can also implement methods and techniques described above toprompt a user to provide information confirming whether this assemblyunit is defective and to label the corresponding vector accordingly. Thesystem can then leverage these feedback from the user to predict similaraesthetic and/or functional outcomes of future assembly units based onproximity to this anomalous assembly unit in the multi-dimensionalfeature space. The system can therefore regularly recalculate clustersand define new clusters as anomalous assembly units are detected andthen labeled by users over time.

The system can implement similar method and techniques to detect an newassembly unit as anomalous—which may be indicative of a defect—based ona distance of a vector representing this new assembly unit to a nearestcluster (e.g., to a centroid of this nearest vector), such as regardlessof whether this nearest cluster is labeled with results or outcomes ofassembly units vectors contained in this cluster.

10.1 Thresholding and Tuning

In this variation, the system can implement thresholds for detectinganomalous assembly units and can refine these thresholds over time basedon feedback provided by a user.

In one example, the system: accesses a new inspection image of a newassembly unit recorded by the optical inspection station duringproduction of the new assembly unit; detects a new set of features inthe new inspection image; generates a new vector representing the newset of features in the multi-dimensional feature space; and flags thenew assembly unit as anomalous if the new vector deviates from a nearestcluster of vectors by more than a preset low threshold distance. In thisexample, if the distance between the new vector and the nearest clusterexceeds a high threshold distance greater than the low thresholddistance in the multi-dimensional feature space, the system can issue aprompt—such as through the optical inspection station—to inspect the newassembly unit for a defect prior to additional assembly of the newassembly unit. However, if the distance between the new vector and thenearest cluster falls between the low threshold distance and the highthreshold distance in the multi-dimensional feature space, the systemcan serve a prompt to a user to manually inspect or test the newassembly unit upon its completion (i.e., rather than immediately).

The system can also access results of this further inspection of the newassembly unit. For example, a first cluster nearest the new vector—thatrepresents the new assembly unit—is associated with non-defectiveassembly units and if an inspection result for the new assembly unitindicates complete functionality of the new assembly unit, the systemcan: increase the (low) threshold distance for identifying assemblyunits as anomalous; and recalculate a boundary around the first clusterto include the new vector. Similarly, if an inspection result for thenew assembly unit indicates presence of a particular defect, the systemcan modify a boundary of a second cluster of vectors associated withthis particular defect to include the new vector; the system can modifythe (low) threshold distance for identifying assembly units as anomaloussuch that this revised threshold would have located this new vector inthe second cluster.

However, the system can implement any other techniques or schema toidentify assembly units as anomalous, to selectively prompt inspectionof these anomalous assembly units, and to revise models for detectinganomalous assembly units based on such inspection results.

10.2 Anomaly Inspection Support

In this variation, once a new assembly unit is identified as anomalous,the system can also assist a user in identifying an anomalous region inthis new assembly unit, which may aid the user in confirming presence orabsence of a defect in the assembly unit.

In one implementation, the system isolates a dominant feature type ofthe new vector (e.g., a particular dimension in the new vector in themulti-dimensional feature space) that exhibits greatest deviation from anearby vector associated with a known outcome. For example, the newvector may align strongly with a vector in a nearby cluster in a largeproportion of dimensions in the feature space. However, feature valuesin a small number of dimensions in the new vector may deviatesignificantly from feature values in the same dimensions in this nearbycluster. The system can therefore flag this small number of dimensionsin the new assembly unit as anomalous. The system can then: locate an“anomalous region of interest” in the new inspection image of the newassembly unit that contains features in this small number of dimensions;and serve the new inspection image to a user portal with the anomalousregion of interest highlighted or otherwise indicated, thereby drawingattention to a region of the new assembly unit that is anomalous. A usercan then provide feedback regarding presence of a defect in thisanomalous region of interest based on visual inspection of the annotatedinspection image; alternatively, the user can reference the annotatedinspection image to guide dismantling, testing, or other physicalinspection of the new assembly unit.

In this variation, the system can also implement methods and techniquesdescribed above to rank new assembly units by strength of deviation fromclusters of vectors representing known outcomes. For example, upondetecting a set of assembly units represented by vectors that deviatefrom established clusters of vectors, the system can: rank this set ofassembly units as a function of distance from their nearest clusters (oras a function of distance from a particular cluster); and then serve aseries of prompts to inspect these assembly units—to a user portal(e.g., executing on a computing device distinct from the opticalinspection station)—as a function of rank. Following receipt ofinspection results from the user in order of rank, the system canrecalculate clusters of vectors in the feature space and refine itsassessment of anomalous vectors accordingly.

However, the system can present visual data representative of ananomalous assembly unit to a user in any other way.

11. Features Predictive of Defect

In another variation shown in FIG. 2, the method S100 includes: labelinga first cluster, in the set of clusters, with a defect indicated ininspection results of a first subset of assembly units, in the set ofassembly units, associated with vectors in the first cluster; labeling asecond cluster, in the set of clusters, with absence of the defect basedon inspection results of a second subset of assembly units, in the setof assembly units, associated with vectors in the second cluster;identifying a model set of features common to vectors in the firstcluster and excluded from vectors in the second cluster in Block S160;and associating the model set of features with the defect in Block S162.Generally, in this variation, the system implements methods andtechniques similar to those described above to isolate featuresindicative of a defect mode in assembly units of the assembly type.

In one implementation, the system groups a population ofvectors—representing assembly units of the assembly type producedpreviously on the assembly line—into a set of (e.g., two or more)clusters in Block S150. The system then retrieves existing outcome datafor these assembly units and/or interfaces with a user to projectoutcomes of a representative subset of these assembly units ontoclusters containing vectors of these representative assembly units. Forexample, for a first cluster of vectors thus identified in the featurespace, the system can: select a first vector representative of the firstcluster (e.g., proximal a centroid of the first cluster); serve a firstprompt to a user, via a user portal, to inspect a first assemblyunit—represented by the first vector—for a defect; and then label thefirst cluster with a particular defect in response to receipt of a firstinspection result indicating presence of the particular defect in thefirst assembly unit. The system can implement similar methods andtechniques to: select a second vector representative of a second cluster(e.g., proximal a centroid of the second cluster) distinct from thefirst cluster in the feature space; serve a second prompt to the user toinspect a second assembly unit—represented by the second vector—for adefect; and then label the second cluster with absence of the particulardefect in response to receipt of a second inspection result thatexcludes an indication of the particular defect in the second assemblyunit. The first and second clusters may thus represent features—in oneor more dimensions—that are predictive of presence and absence,respectively, of the particular defect.

The system can then implement methods and techniques similar to thosedescribed above to isolate a dominant feature type characteristic ofvectors in the first cluster and that exhibit significant deviation fromvectors in the second cluster. For example, the system can: calculate afirst composite vector representing the first cluster (e.g., an averageof vectors in the first cluster), which may contain featuresrepresentative of the particular defect; and calculate a secondcomposite vector of the second cluster, which may contain featuresrepresentative of absence of the particular defect. In this example, thefirst composite vector may align strongly with (e.g., contain similarfeature values as) the second composite vector in a large proportion ofdimensions in the feature space. However, feature values in a smallnumber of dimensions in the first composite vector may deviatesignificantly from feature values in the same dimensions in the secondcomposite vector. The system can therefore flag this small number ofdimensions as exhibiting strong correlation to the particular defect.

The system can then: automatically locate a region of interest—in afirst inspection image of a first assembly unit representative of thefirst cluster—that contains features in this small number of dimensions;and serve this first inspection image to a user portal with theanomalous region of interest highlighted or otherwise indicated, therebydrawing attention to a region of the first assembly unit that may be thesource of the particular defect. A user can then provide feedbackregarding presence of a defect in this region of interest in the firstinspection image of the first assembly unit based on visual inspectionof this annotated inspection image; alternatively, the user canreference the annotated inspection image to guide dismantling, testing,or other physical inspection of the first assembly unit to confirmwhether features in these isolated dimensions exhibit strong correlationto the particular defect. The system can then update or modify acorrelation between this small set of dimensions and the particulardefect based on this feedback in order to construct a model predictiveof the particular defect in Block S162.

Furthermore, pending confirmation from the user that this region ofinterest in the first assembly unit exhibited the particular defect ormay otherwise contribute to the particular defect, the system can promptselective inspection of additional assembly units in and around thefirst and second clusters in order to further refine features and/ordimensions predictive of the particular defect. In the foregoingexample, the system can: identify a first feature (e.g., a value orvalue range in a first dimension) and a second feature (e.g., a value orvalue range in a second dimension) common to vectors in the firstcluster and excluded from vectors in the second cluster. The system canthen: identify a third vector that represents a third assembly unit,contains the first feature, and excludes the second feature; generate aprompt to inspect the third assembly unit for the particular defect; andthen disassociate the first feature from the defect if the user thusconfirms absence of the particular defect from the third assembly unit.The system can therefore select key assembly units characterized bycertain combinations of features, issue prompts to selectively label orprovide feedback for these assembly units, and then construct a robustmodel for the particular defect (or for a set of known defects) based onthis minimum targeted input from one or more users.

The system can then leverage this model and inspection images of pastand/or future assembly units to identify assembly units highly likely toexhibit the particular defect and then serve intelligent prompts toinspect (or cull, or rework) these assembly units accordingly. Forexample, the system can: receive a new inspection image of a newassembly unit recorded by the optical inspection station; detecting anew set of features in the new inspection image; and then serve a promptto inspect the new assembly unit for the particular defect—such as tothe optical inspection station or to a local computing device near theoptical inspection station—in (near) real-time in response to the newset of features approximating the model set of features thus associatedwith the particular defect.

The system can implement similar methods and techniques to associatedifferences between two or more distinct clusters of vectors in thefeature space with various positive and negative outcomes and toconstruct models of representative feature sets accordingly.

11.1 Example: Antenna Malfunction Model

In one example, a batch of assembly units are assembled; during testing,a first set of assembly units in this batch are determined to exhibitantenna malfunction, and a second set of assembly units in this batchare determined to exhibit proper antenna function. In this example, thesystem: generates vectors from inspection images of these assembly unitsin Blocks S120, S122, and S124; retrieves antenna test data for theseassembly units; tags each vector with the antenna function of itscorresponding assembly unit in Block S140; and implements structureddata analysis techniques to group these vectors into two (or more)clusters in Block S130. The system then: identifies a first cluster ofvectors representing all or some assembly units in the first set;identifies a second cluster of vectors representing all or some assemblyunits in the second set; identifies a first set of feature rangescontaining vectors in the first cluster but disjointed from (i.e., notcontaining) vectors in the second cluster; and associates the first setof feature ranges with antenna malfunction. For example, the system canimplement a support vector machine to define the first set of featureranges that includes: features (e.g., a fastener, fillet, trace, circuitcomponent) missing entirely from either the first or second cluster; arange of component spacings; a range of relative component orientations;and/or a range of feature geometries (e.g., shape, length, profile,surface finish, etc.); etc.

A combination of all or a subset of feature ranges in the first set maytherefore be indicative or predictive of antenna failure in assemblyunits of this type at this production stage. In particular, the systemcan link a specific subset of feature ranges—filtered from a largenumber (e.g., “n”) of features extracted from inspection images ofdefective and sound assembly units—to a particular defect. The systemcan then predict a similar defect in a second assembly unit responsiveto the first set of feature ranges containing a vector representing thissecond assembly unit.

Similarly, in the foregoing example, the system can: identify a secondset of feature ranges containing vectors in the second cluster butdisjointed from (i.e., not containing) vectors in the first cluster; andassociate the second set of feature ranges with proper antenna function.The system can thus link a specific subset of features—from a largenumber of features extracted from inspection images of defective andsound assembly units—to a proper aesthetic condition or proper functionof an assembly unit of this type at this production stage. The systemcan then predict proper aesthetic condition or function in a secondassembly unit responsive to the second set of feature ranges containinga vector representing this second assembly unit. Alternatively, if thesecond set of feature ranges fails to contain some or much of the secondvector, the system can predict similar failure of this second assemblyunit but also prompt an engineer to inspect select regions of thissecond assembly unit in which these features of the second assembly unitfall outside of the second set of feature ranges.

In this variation, by defining the first set of feature ranges, thesystem can filter a total number of features representing an assemblyunit down to a much smaller number of features likely to indicate afunctional or aesthetic defect; reducing this number of features down toa number digestible by an human enables the system to collect feedbackfrom a user, such as in the form of an hypothesis regarding whichfeature ranges in this set contributed to failure and/or in the form oflabels for these feature ranges (e.g., “part missing,” “broken trace,”“misoriented electrical component,” etc.).

In one implementation, the system: selects a first inspection imagecorresponding to a vector representative of the first cluster;highlights features in the first inspection image corresponding to thefirst set of feature ranges; serves the first inspection image to a userthrough an instance of the user portal; and prompts the user to manuallyselect and label these highlighted features. In the foregoing example,the system can: prompt the user to manually select highlighted featuresshe predicts as the root cause of antenna failure in the correspondingassembly unit.

Specifically, in the foregoing example, by identifying the first set offeature ranges common to assembly units exhibiting antenna malfunctionbut not characteristic of assembly units exhibiting proper antennafunction, the system can filter a large number (e.g., thousands, or “n”)of features down to a significantly smaller number of featuresexhibiting greater likelihood of representing a defect mode for antennasin this assembly. By prompting the user to review this much smaller setof features or to review inspection images containing a spectrum ofranges of these features, the system can enable the user to relativelyquickly identify or predict a root cause of antenna malfunction in thefirst set of assembly units and to label these features and/orinspection images accordingly. The system can then automaticallyidentify other assembly units in the same batch or in other batches thatmay exhibit a similar antenna defect mode—with a high degree of accuracyand before antennae in these assembly units are tested electronically—byidentifying vectors corresponding to these assembly units and exhibitingfeatures within the first set of feature ranges described above.

11.2 User Portal+UX

In this variation, the system can also: select one or a subset ofvectors representative of each of a set of discrete clusters; retrieveone inspection image corresponding to each of these vectors; serve theseinspection images to the user for review in parallel or in series thoughthe user portal; prompt the user to label specific features in theseinspection images that the user judges, expects, or hypothesizes tocontribute to success and failure of an aspect of these units; and buildand refine a classifier for features based on these user-suppliedlabels.

In the foregoing example, the system can: retrieve inspection imagescorresponding to vectors representative of clusters labeled with antennafailure, adequate antenna function, superior antenna function, and/orother antenna-related outcomes; align these inspection images by commonfeature (e.g., PCB corners, housing edges, etc.), such as described inU.S. patent application Ser. No. 15/407,158; and serve these inspectionimages to a user through the user portal. The user can then: scrollthrough these inspection images within the user portal to discern visualdifferences between assembly units represented by these inspectionimages; select a region in one or a set of these inspection images thatthe user judges, expects, or hypothesizes to have contributed to antennafailure and/or success across these assembly units, such as by drawing avirtual box around this region of an inspection image or by dropping apointer (e.g., a flag) over this region of the inspection image; andthen enter a manual label linking this region or pointer to antennafunctionality. In this implementation, the system can then associatefeatures contained within this selected region or features intersectingthe user-defined pointer with known antenna function outcomes ofcompleted assembly units of this assembly type. Therefore, the systemcan: assemble a group of inspection images associated with vectorsrepresentative of different clusters labeled with different outcomes ofa particular function or aesthetic (e.g., antenna function, buttonfunction, camera function, battery life, aesthetic surface quality,etc.); serve these inspection images in series or in parallel to theuser; and receive—from the user—manual indications of regions withinthese inspection images that correspond to these functional or aestheticoutcomes (e.g., whether the antenna functions, whether the buttonfunctions, whether the camera functions, whether the assembly unitexhibits proper battery life, whether a fastener is present, whetherrelative positions of two features fall inside or outside of anacceptable bound, etc.). The system can then train a classifier toassociate these functional or aesthetic labels with absolute (e.g.,binary) or relative (e.g., spectrum) features extracted from theseregions or extracted near these pointers—defined by the user—across thisset of inspection images.

In another implementation, the system can: assemble a group ofinspection images associated with vectors representative of differentclusters; and then highlight regions in each of these inspection imagescontaining features correlated with (e.g., representing) differencesbetween these vectors. For example, the system can insert colored boxesover these regions of these inspection images and locate these boxed bycommon visual features (e.g., an edge, two corners) shared across theseinspection images. Alternatively, the system can crop these inspectionimages to these regions or automatically zoom into these regions as theuser scrolls through these inspection images within the user portal. Thesystem can thus align these inspection images and serve these inspectionimages to the user through the user portal, as described above. As theuser views these inspection images (e.g., in series) within the userportal, the user can manually label regions in select inspection imageswith certain functional or aesthetic outcomes, such as “antennafailure,” “adequate antenna function,” “superior antenna function,”“missing fastener,” “fastener intact,” “proper component alignment,”“components rotated beyond acceptable tolerance,” or “components offsetbeyond acceptable tolerance,” etc. The system can then train theclassifier to associate these functional or aesthetic outcomes withabsolute or relative features extracted from these regions—defined bythe user—across this set of inspection images.

In the foregoing implementations, the system can also implement theclassifier to automatically define regions of inspection imagespredicted to correlate to certain known outcomes (e.g., a binary outcomesuch as presence of a fastener, or a spectrum outcome such as degree ofoffset between two components that affect a particular function) andwrite predicted outcome labels to these regions of these inspectionimages. The system can then serve these inspection images—with theseregions highlighted or cropped—to the user for confirmation of thesepredicted outcome labels; the system can then update or refine theclassifier accordingly.

In another implementation, the system can assemble a group of inspectionimages exhibiting features spanning a range (i.e., a “spectrum”) offeature values. For example, the system can aggregate a group ofinspection images corresponding to vectors spanning two clusters in oneor a subset of feature dimensions. The system can then: highlight orcrop regions of these inspection images containing featurescorresponding to this subset of feature dimensions; align these regionsof these inspection images by common features, as described above; andserve these aligned image regions to the user in series through the userportal, such as in order of feature value (e.g., length). Whilescrolling through these image regions in order within the user portal,the user can label or demarcate: a sequence of these image regions inwhich represented areas on assembly units are within prescribedtolerances; another sequence of these image regions in which representedareas on assembly units are not within prescribed tolerances; and/or asequence of these image regions in which represented areas on assemblyunits are near prescribed tolerances but require further testing orinspection to confirm functionality. Based on labels thus supplied bythe user, the system can distinguish clusters of vectors representingassembly units within prescribed tolerances from clusters of vectorsrepresenting assembly units outside of prescribed tolerances. The systemcan then predict success of this function in future assembly units basedon proximity of their representative vectors to the clusters.

11.3 Supervised Machine Learning

In the foregoing implementations, the system can assist a user inproviding supervision by selecting and packaging image data ofrepresentative assembly units; the system can then implement supervisedmachine learning techniques to develop a classifier (e.g., a model) forcorrelating features extracted from inspection images of assembly unitswith certain functional and/or aesthetic outcomes over time.

The system can also update (or “train”) the feature classifier describedabove to place greater weight or priority on detection and extraction offeatures represented in the first set of feature ranges in order toincrease sensitivity of the system to detecting a defect represented bythis first set of feature ranges.

Furthermore, the system can implement similar methods and techniques tolink other discrete clusters of vectors—associated with a particulardefect—with other defect modes for this defect and to define sets offeature ranges for these other defect modes.

As the user (or multiple users) enters feedback and label soundness anddefects of assembly units and/or as test data is generated, the systemcan repeat the foregoing methods and techniques to recalculate clustersof vectors, to identify trends in soundness and defects of assemblyunits represented by these vectors, and to refine soundness and defectlabels assigned to these clusters accordingly in order to improvereal-time prediction of known defects in new assembly units and toimprove asynchronous detection of newly-identified defects in existingassembly units based on inspection images of these assembly units.

11.4 Features Predictive of Defect Absence

In a similar variation, the system can: label a first cluster, in theset of clusters, with a defect indicated in inspection results of afirst subset of assembly units, in the set of assembly units, associatedwith vectors in the first cluster, the defect corresponding to aparticular function of the particular assembly type; label a secondcluster, in the set of clusters, with absence of the defect based oninspection results of a second subset of assembly units, in the set ofassembly units, associated with vectors in the second cluster; identifya model set of features common to vectors in the second cluster andexcluded from vectors in the first cluster; and associate the model setof features with proper operation of the particular function for theparticular assembly type.

In particular, in this variation, the system can implement methods andtechniques similar to those described above to coverage on featurevalues (or feature ranges) in a small number of dimensions—in thefeature space—that exhibit strong correlation to absence of one or moreknown defects in an assembly unit.

12. Yield Protection: Drift Detection

In one variation shown in FIG. 6, the system tracks trends in vectorposition over multiple dimensions in order to predict changes in yieldduring production of assembly units in Block S170. In particular, bygenerating multi-dimensional vectors representing multiple features ofassembly units produced over time and extrapolating trends in thesevectors relative to past vectors representing assembly units of knownoutcomes, the system can identify drift of certain features (e.g.,dimension, tolerance, geometry, etc.) in assembly units at theproduction stage over time. If left unaddressed, such drift mayeventually lead to defective assembly units and decreased yield on theassembly line. Therefore, as the system detects such drift at higherlevels of confidence over time, the system can issue prompts to inspectkey segments of the assembly for sources of this drift well before thisdrift results in decreased yield.

12.1 Drift Toward Known Defect

In one implementation, the system: generates a first set of vectors froma first sequence of inspection images of a first set of assembly unitsassembled over a first period of time in Block S124; groups this firstset of vectors into a set of clusters in Block S130; and labels a firstcluster in this set with a particular defect based on a known outcome ofa first assembly unit represented by a first vector contained in thefirst cluster in Block S140, as described above. The system can then:access a second sequence of timestamped inspection images of a secondset of assembly units of the same assembly type and produced over asecond period of time (e.g., the last hour, day, week, or monthsucceeding the first period of time); generate a second set oftimestamped vectors from this second sequence of inspection images; andcalculate strength of a trend of this second set of vectors toward thefirst cluster—associated with the particular defect—over this secondperiod of time in Block S170. If the system thus identifies a trendtoward the first cluster over time—such as a strong trend of theserecent vectors toward the first cluster rather than a randomdistribution of distances between recent vectors and the first clusterover the second period of time—the system can predict manufacturingdrift on the assembly line leading up to the optical inspection stationthat recorded these inspection images and prompt a user (e.g., amanufacturing engineer, an assembly line technician, etc.,) to inspect asegment of the assembly line preceding this optical inspection stationfor a root cause of the drift. The system can thus notify a user ofmanufacturing drift along the assembly line toward featurescharacteristic of a particular known defect well before an increase inthe frequency of assembly units exhibiting this particular defectoccurs.

In one example, the system can implement the foregoing method andtechniques to generate a vector for each subsequent inspection image ofa new assembly unit received from an optical inspection station during aproduction run. The system can then implement a k-nearest neighborclassifier to locate the new vector in one of: a first clusterassociated with realization of a defect; and a second cluster associatedwith avoidance of the defect. The system can then: calculate a temporalmulti-dimensional trendline that represents locations of these newvectors in the feature space; determine whether the trendline isdirected toward the first cluster (toward another cluster associatedwith another defect, and/or away from the second cluster); and extract arate of drift toward the first cluster (toward another clusterassociated with another defect, and/or away from the second cluster)from the trendline. If this trendline is directed toward the firstcluster (or otherwise away from the second cluster), if the rate ofdrift exceeds a threshold rate (e.g., a threshold drift distance perassembly unit), and/or if the trendline has reached a threshold distancefrom a boundary (or from the centroid) of the first cluster in thefeature space, the system can flag the assembly line for inspection,such as a segment of the assembly line just preceding the opticalinspection station that recorded these inspection images.

Based on this flag, an engineer or operator may then inspect theassembly line or production processes for a change that yielded thisdrift toward features correlated with this defect. The system can thusalert and/or assist the engineer or operator in discovering andcorrecting production issues before these issues affect yield at theassembly line. Over time, the system can repeat this process for eachnew inspection image that represents a new assembly unit of this sameassembly type at the same production stage and imaged in the sameorientation. For example, the system can: derive clusters representingpositive and negative outcomes in hundreds, thousands, or millions ofassembly units completed on the assembly line; detect drift in features(or feature values) present in tens or hundreds of new assembly unitsproduced on the assembly line; and notify a user at or affiliated withthe assembly line of such drift detected over this relatively smallsequence of recent assembly units.

Therefore, the system can: calculate a rate of drift of a second set ofvectors—representing a recent sequence of assembly units—from a secondcluster associated with a positive outcome (e.g., lack of any defect orlack of a particular defect) and/or toward a first cluster associatedwith a negative outcome (e.g., presence of one or more defects) over aperiod of time; and then generate a notification or prompt to inspectthe assembly line in response to this rate of drift exceeding a presentthreshold. The system can also calculate an urgency for inspection ofthe assembly line: based on (e.g., proportional to) the rate of drift ofthese vectors outwardly from the second cluster and/or toward the firstcluster; based on (e.g., inversely proportional to) a proximity ofvectors representing recent assembly units to a border of the firstcluster; and/or based on (e.g., inversely proportional to) residuals ofthe variance between these recent vectors and the trendline; etc. Thesystem can then incorporate a quantitative or qualitative indicator ofthis urgency in the prompt in order to indicate to the user how quicklythe assembly line may require addressing before a decrease in yieldoccurs.

(In an example similar to the example above, for a particular defectmode (e.g., a functional or aesthetic requirement, etc.) defined for anassembly in a particular production stage, the system can define andstore a set of feature ranges correlated with avoidance of this defectmode, as described above. (The system can additionally or alternativelydefine and store a range or a set of feature ranges correlated withrealization of this defect mode, as described above.) In this example,as an inspection image of each subsequent assembly unit in this sameproduction stage is received over time, the system can: extract these(and other) features from the inspection image; generate a new vectorbased on these features; locate the new vector relative to past vectors;and calculate a trend in the location of the new and past vectors overtime over multiple (e.g., “n-number” of) dimensions. In particular, thesystem can process the presence and dimensions of these features over asequence of inspection images—corresponding to a sequence of assemblyunits—to determine whether any of these features is trending toward aboundary of an acceptable range correlated with avoidance of this defectmode. (Similarly, the system can determine whether any of these featuresis trending toward a range correlated with realization of this defectmode.) If so, the system can flag the assembly line or flag productionprocesses related to this functional or aesthetic requirement.)

12.2 Drift Correction Confirmation

In this variation, the system can also confirm that a change along theassembly line (or upstream of the assembly line)—following issuance of anotification of drift in Block S170—has resulted in correction orreduction of this detected drift. In particular, by repeating theforegoing methods and techniques, the system can detect: inversion of aprevious trend such that vectors representing new assembly units trendback toward a cluster associated with a positive outcome; a reduction inthe rate of the previous trend toward a cluster associated with anegative outcome; or elimination of the previous trend and a return torandom distribution of (most) vectors representing new assembly units inand around a cluster associated with a positive outcome.

For example, the system can generate the prompt to inspect productionpreceding the optical inspection station along the assembly line inresponse to detecting a strong trend of a second set ofvectors—representing a second sequence of assembly units—toward a firstcluster associated with a negative outcome over a period of time inBlock S170, as described above. Later, the system can: access a thirdsequence of inspection images of a subsequent set of assembly unitsrecorded by the optical inspection station over a next period of time;detect features in each new inspection image and transform thesefeatures into vectors representing the subsequent set of assembly units;and confirm correction of a segment of the assembly preceding theoptical inspection station in response to detecting a weak trend of thisthird set of vectors toward the first cluster over this period of time.

When such trend correction is thus detected, the system can clear aprompt or notification to correct the assembly line. However, the systemcan implement any other method or technique to confirm and respond tosuch correction of the assembly line.

12.4 Drift Away Known Defect Absence

In a similar variation, the system can implement similar methods andtechniques to detect and respond to a trend amongst vectors—representingrecent assembly units—away from a cluster associated with a positiveoutcome (i.e., rather than necessarily toward a cluster associated witha negative outcome).

In one implementation, the system: calculates a centroid of a clusterassociated with a positive outcome (e.g., no defects detected or noterminal defects detected); calculates distances between vectorsrepresenting new assembly units and this centroid in the feature space;and then calculates a rate of change of this distance across thissequence of vectors of time. If this rate of change is positive, if thisrate of change exceeds a threshold rate, and/or if an average of thesedistances over a recent sequence of vectors exceeds a thresholddistance, the system can determine that production on the assembly lineis trending generally away from assembly units with positiveoutcomes—even if yield at the assembly line has not changed—and thengenerate a prompt to inspect the assembly line accordingly in BlockS170.

13. Design Tools

In another variation, the system guides a user in identifying: relativedimensions (e.g., “geometric dimensions”) and/or dimensional variations(i.e., “tolerances”) of features that yield sound units; and otherrelative dimensions and/or dimensional variations of features that yielddefective assembly units. Generally, the system can leverage labeledvectors, clusters of similar vectors, and feature sets representative ofthese clusters, as described above, to associate real dimensions andtolerances of real features of an assembly type that avoid knowndefects. In particular, by representing assembly units asmulti-dimensional vectors containing relatively large numbers of uniquefeatures and generally without bias or understanding of the context ofthese features, the system can rapidly extract deeper, higher-orderinsights into which features predict defects and ranges of thesefeatures that still yield sound assembly units. The system can thenautomatically—or with the assistance of the user—transform these featureranges into dimension and tolerance suggestions for the assembly inorder to reduce production costs and/or increase yield.

In one implementation, the system characterizes a set of feature rangesthat contain a cluster of vectors labeled as sound (i.e., meeting allaesthetic and functional requirements), such as by implementing methodsand techniques described above. For example, the system can isolateregions of inspection images representing the set of feature ranges,align these regions of these inspection images by a common feature,present these image regions in series through the user portal, andpermit the user to scroll through these image regions in sequence,thereby visually indicating to the user ranges of dimensions, profiles,and orientations, etc. of features that still yield sound assemblyunits. The system can further transform these feature ranges into thereal nominal distances, angles, profiles, and tolerances for individualfeatures and groups (or “stacks”) of features in assemblies of thistype. Alternately, the system can enable the user to select a featurefor measurement extraction (e.g., a distance or angle between twocorners or edges), and the system can extract a range of dimensions (ora nominal dimension and tolerance) from the image regions, such asdescribed in U.S. patent application Ser. No. 15/407,158. The user(e.g., an engineer) may then adjust a nominal dimension and/or toleranceon a predefined dimension of this feature for this assembly typeaccordingly. For example, if the system determines—from a cluster ofvectors of sound assembly units—that location of two components(represented by multiple features) in the assembly type to within onemillimeter of a nominal distance and to within 5° of a nominal anglestill yields a sound product, whereas these dimensions were originallyassigned tolerances of 0.1 millimeter and 1°, respectively, the systemcan guide the user in loosening these tolerances on relative (e.g.,geometric) location of these two components, which may reduce productcosts for this assembly type without significantly reducing yield.Conversely, if the system determines—from a cluster of vectors of soundassembly units—that location of two components (represented by multiplefeatures) in the assembly type by more than 0.1 millimeter beyond anominal distance and by more than 1° from a nominal angle yields adefective product, whereas these dimensions were originally assignedtolerances of 0.2 millimeter and 2°, respectively, the system can guidethe user in tightening these tolerances on relative location of thesetwo components in order to increase yield. The system can implementsimilar methods and techniques to guide the user in adjusting a nominaldimension specified for the type of assembly.

Similarly, the system can assist the user (or a machine) in refininggeometry dimensioning of the type of assembly. For example, a set offeature ranges—identified by the system predictive of a sound assemblyunit—can define acceptable ranges of second-order features, such asincluding a relative distance, angle, or profile between two discretefeatures. The system can indicate importance of the relative distance,angle, or profile, etc. between these two features in achieving a soundassembly unit; and the user can define a datum and set a dimension fromthis datum to another feature for the assembly accordingly, such as toimprove yield.

In the variation above in which the system develops a model set offeatures predictive of presence or absence of a defect), the system can:detect a second range of a particular feature—in the model set offeatures—represented in a second cluster of vectors associated with apositive outcome and that is distinct from a first range of thecorresponding feature represented in a first cluster of vectorsassociated with negative outcomes; and associate the second range of theparticular feature in the model set of features with a positive outcome(e.g., sufficient operation of a particular function that is defectivein assembly units represented in the first cluster). The system canthen: identify a subset of existing vectors in the second cluster thatare representative of this range of the particular feature; locate theparticular feature in a subset of inspection images of assembly unitscorresponding to this subset of vectors; transform differences inlocations of the particular feature across this subset of inspectionimages into a dimension; and then store this dimension as amanufacturing tolerance in this region of the assembly units of thistype. Alternatively, the system can serve this subset of inspectionimages to a user via the user portal, such as in sequence or in acomposite image, and the user can manually extract a dimension range(e.g., a “tolerance”) for a particular feature from these inspectionimages. The user can then update engineering drawings or an assemblyspecification, etc. to reflect this dimension range, such as bytightening or loosening an assembly specification to reflect thisdimension range thus correlated with positive outcomes for assemblyunits produced on the assembly line.

Therefore, the system can guide a user in adjusting dimension andtolerance specifications of the assembly in order to increase yieldand/or decrease production cost based on features extracted frominspection images of past assembly units and related test data for theseassembly units during production, such as after a first batch of 30assembly units are produced or even after 30M assembly units areproduced.

The system can implement similar methods and techniques to guide a userin understanding the affects of dimensions and tolerances assigned toindividual features and groups of features on incidence of defects. Forexample, when designing a new assembly in which a particular defect isconceivable, an engineer may interface with the system to aggregate setsof feature ranges that predict this particular defect in a previousassembly in order to ascertain dimensions, tolerances and datums likelyto avoid this particular defect in production of the new assembly.Therefore, the system can implement these methods and techniques toprovide both production management tools and new design tools forengineers, assembly line operators, assembly workers, etc.

14. Sub-Clustering

In one variation, the system can implement the foregoing methods andtechniques to identify and label sub-clusters of vectors within onecluster of vectors (and to identify and label sub-sub-clusters ofvectors within one sub-cluster of vectors, etc.). For example,high-level clusters may be dominated by large differences betweenassembly units (e.g., presence or lack of a large component withinassembly units), though relevant variations may also exist withinassembly units represented by vectors within a cluster, and thesevariations may indicate success of failure of certain functional and/oraesthetic aspects of these assembly units. Therefore, the system can:implement methods and techniques described above to distinguish two ormore first-level clusters—among a set of vectors generated frominspection images of assembly units passing through an opticalinspection station over time—containing vectors exhibiting large-scaledifferences; select a particular cluster in this set of first-levelclusters; and then repeat the foregoing methods and techniques todistinguish two or more second-level clusters—within the particularcluster—containing vectors exhibiting smaller-scale differences. Thesystem can then detect or predict failures, explore relationshipsbetween failures and nominal dimensions and tolerances on thesedimensions, and/or detect anomalies across assembly units represented byvectors contained in these second-level clusters, etc., as describedabove.

15. Simultaneous Assembly Line Tracking Modes

The system can also implement some or all of the foregoing variations ofthe method S100 simultaneously to detect defects in past assembly units,to detect defects in a new unit, to characterize features predictive ofpositive and/or negative outcomes, and/or detect drift in features ofassembly units produced on the assembly line over longer time scales andto selectively inform or assist various users of the defects, features,and/or trends. For example, the system can simultaneously executereal-time yield protection, real-time anomaly detection, defectpropagation, and trend detection techniques described herein.

The systems and methods described herein can be embodied and/orimplemented at least in part as a machine configured to receive acomputer-readable medium storing computer-readable instructions. Theinstructions can be executed by computer-executable componentsintegrated with the application, applet, host, server, network, website,communication service, communication interface,hardware/firmware/software elements of a user computer or mobile device,wristband, smartphone, or any suitable combination thereof. Othersystems and methods of the embodiment can be embodied and/or implementedat least in part as a machine configured to receive a computer-readablemedium storing computer-readable instructions. The instructions can beexecuted by computer-executable components integrated bycomputer-executable components integrated with apparatuses and networksof the type described above. The computer-readable medium can be storedon any suitable computer readable media such as RAMs, ROMs, flashmemory, EEPROMs, optical devices (CD or DVD), hard drives, floppydrives, or any suitable device. The computer-executable component can bea processor but any suitable dedicated hardware device can(alternatively or additionally) execute the instructions.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the embodiments of the invention without departing fromthe scope of this invention as defined in the following claims.

We claim:
 1. A method for predicting manufacturing defects, the methodcomprising: accessing a set of inspection images of a set of assemblyunits, of a particular assembly type, recorded by an optical inspectionstation during production of the set of assembly units; for eachinspection image in the set of inspection images: detecting a set offeatures in the inspection image; and generating a vector, in a set ofvectors, representing the set of features in a multi-dimensional featurespace; grouping neighboring vectors, in the set of vectors, in themulti-dimensional feature space into a set of vector groups; and inresponse to receipt of a first inspection result indicating a defect ina first assembly unit, in the set of assembly units, associated with afirst vector in a first vector group, in the set of vector groups:labeling the first vector group with the defect; calculatingprobabilities of the defect in assembly units in the set of assemblyunits based on proximity of corresponding vectors to the first vector inthe multi-dimensional feature space; flagging a second assembly unitassociated with a second vector, in the first vector group, asexhibiting characteristics of the defect; and generating a prompt toinspect the second assembly unit in response to a probability of thedefect in the second assembly unit exceeding probabilities of the defectin the other assembly units in the set of assembly units.
 2. The methodof claim 1, wherein detecting sets of features in the set of inspectionimages comprises, for each inspection image in the set of inspectionimages: dividing the inspection image into a set of image segments;selecting a subset of image segments, in the set of image segments,corresponding to an assembly unit region of interest associated with theoptical inspection station, in a set of optical inspection stationsarranged along an assembly line; for each image segment in the subset ofimage segments, extracting a subset of features from the image segment;and compiling subsets of features extracted from the subset of imagesegments into the set of features in the inspection image.
 3. The methodof claim 2: wherein accessing the set of inspection images comprisesaccessing the set of inspection images from a database, the set ofinspection images recorded over a first period of time by the opticalinspection station; further comprising: receiving a second inspectionimage of the second assembly unit recorded by the optical inspectionstation at a second time succeeding the first period of time; dividingthe second inspection image into a second set of image segments;selecting a second subset of image segments, in the second set of imagesegments, corresponding to the assembly unit region of interestassociated with the optical inspection station; extracting a subset offeatures from each image segment in the second set of image segments;compiling subsets of features extracted from the second subset of imagesegments into a second set of features in the second inspection image;and generating the second vector representing the second set of featuresin the multi-dimensional feature space; and wherein flagging the secondassembly unit comprises serving a prompt, to inspect the second assemblyunit for the defect, to a user portal at approximately the second timein response to associating the second vector with the first vector groupbased on proximity of the second vector to the first vector in themulti-dimensional feature space.
 4. The method of claim 3, whereinserving the prompt to inspect the second assembly unit to the userportal comprises serving the prompt to the user portal executing on amobile computing device associated with an operator and located proximalthe assembly line at the second time.
 5. A method for predictingmanufacturing defects, the method comprising: accessing a set ofinspection images of a set of assembly units, of a particular assemblytype, recorded by an optical inspection station during production of theset of assembly units; for each inspection image in the set ofinspection images: detecting a set of features in the inspection image,wherein detecting sets of features in the set of inspection imagescomprises, for each inspection image in the set of inspection images:projecting the region of interest from the first inspection image ontothe inspection image; and extracting the set of features from the regionof interest in the inspection image; and generating a vector, in a setof vectors, representing the set of features in a multi-dimensionalfeature space; and grouping neighboring vectors, in the set of vectors,in the multi-dimensional feature space into a set of vector groups,wherein grouping vectors into vector groups comprises grouping vectors,in the set of vectors, into the first vector group based on proximity tothe first vector in the multi-dimensional feature space; in response toreceipt of a first inspection result indicating a defect in a firstassembly unit, in the set of assembly units, associated with a firstvector in a first vector group, in the set of vector groups: labelingthe first vector group with the defect, wherein labeling the firstvector group with the defect comprises predicting presence of the defectin a subset of assembly units, in the set of assembly units, associatedwith vectors in the first vector group; and flagging a second assemblyunit associated with a second vector, in the first vector group, asexhibiting characteristics of the defect; and serving a first inspectionimage of the first assembly unit to a user portal; and receiving manualselection of a region of interest in the first inspection image via theuser portal, the region of interest in the first inspection imagepredicted by a user to depict a source of the defect.
 6. The method ofclaim 5, further comprising: receiving the first inspection resultcomprising indication of the defect in the first assembly unit that waspreviously vended and returned as defective; and automaticallygenerating a list of serial numbers of the particular assembly type, thelist of serial numbers corresponding to assembly units in the subset ofassembly units previously vended.
 7. The method of claim 5: furthercomprising receiving the first inspection result comprising indicationof the defect comprising an aesthetic defect; and wherein flagging thesecond assembly unit comprises: projecting the region of interest fromthe first inspection image onto a second inspection image of the secondassembly unit; serving the region of interest in the second inspectionimage to a user through a user portal; and prompting the user to confirmpresence of the aesthetic defect in the region of interest in the secondinspection image.
 8. A method for predicting manufacturing defects, themethod comprising: accessing a set of inspection images of a set ofassembly units, of a particular assembly type, recorded by an opticalinspection station during production of the set of assembly units; foreach inspection image in the set of inspection images: detecting a setof features in the inspection image; and generating a vector, in a setof vectors, representing the set of features in a multi-dimensionalfeature space; grouping neighboring vectors, in the set of vectors, inthe multi-dimensional feature space into a set of vector groups, whereingrouping neighboring vectors into the set of vector groups comprises:defining a manifold delineating a first multi-dimensional volumecontaining the first vector and the second vector from a secondmulti-dimensional volume containing a third vector; and grouping vectorswithin the first multi-dimensional volume into the first vector group;in response to receipt of a first inspection result indicating a defectin a first assembly unit, in the set of assembly units, associated witha first vector in a first vector group, in the set of vector groups:labeling the first vector group with the defect; and flagging a secondassembly unit associated with a second vector, in the first vectorgroup, as exhibiting characteristics of the defect, wherein flagging thesecond assembly unit for the defect comprises generating a prompt toinspect the second assembly unit for the defect; receiving confirmationof absence of the defect in the second assembly unit responsive to theprompt to inspect the second assembly unit; in response to confirmationof absence of the defect in the second assembly unit, calculating arevised manifold delineating a first revised multi-dimensional volumecontaining the first vector and a second revised multi-dimensionalvolume containing the second vector; grouping vectors within the firstrevised multi-dimensional volume into a revised first vector group; andgenerating a second prompt to inspect a fourth assembly unit associatedwith a fourth vector, in the revised first vector group, for the defect.9. A method for predicting manufacturing defects, the method comprising:accessing a set of inspection images of a set of assembly units, of aparticular assembly type, recorded by an optical inspection stationduring production of the set of assembly units; for each inspectionimage in the set of inspection images: detecting a set of features inthe inspection image; and generating a vector, in a set of vectors,representing the set of features in a multi-dimensional feature space;grouping neighboring vectors, in the set of vectors, in themulti-dimensional feature space into a set of vector groups; labeling afirst vector group, in the set of vector groups, with a defect indicatedin inspection results of a first subset of assembly units, in the set ofassembly units, associated with vectors in the first vector group, thedefect corresponding to a particular function of the particular assemblytype; labeling a second vector group, in the set of vector groups, withabsence of the defect based on inspection results of a second subset ofassembly units, in the set of assembly units, associated with vectors inthe second vector group; identifying a model set of features common tovectors in the second vector group and excluded from vectors in thefirst vector group; and associating the model set of features withproper operation of the particular function for the particular assemblytype.
 10. The method of claim 9: wherein identifying the model set offeatures comprises detecting a range of a particular feature in themodel set of features represented in the second vector group anddistinct from a range of a corresponding feature represented in thefirst vector group; wherein associating the model set of features withsufficient operation of the particular function comprises associatingthe range of the particular feature in the model set of features withsufficient operation of the particular function; and further comprising:identifying a subset of vectors in the second vector grouprepresentative of the range of the particular feature; locating theparticular feature in a subset of inspection images of assembly unitscorresponding to the subset of vectors; transforming differences inlocations of the particular feature in the subset of inspection imagesinto a dimension; and storing the dimension as a manufacturingtolerance.
 11. The method of claim 9: wherein accessing the set ofinspection images comprises accessing the set of inspection imagescomprising digital color photographic images recorded by the opticalinspection station over a first period of time; and further comprising:receiving a second inspection image of a second assembly unit recordedby the optical inspection station at a second time succeeding the firstperiod of time; detecting a second set of features in the secondinspection image; and in response to the second set of featuresdiffering from the model set of features, serving a prompt to inspectthe second assembly unit to a user portal at approximately the secondtime, the user portal executing on a computing device proximal theoptical inspection station.
 12. A method for predicting manufacturingdefects, the method comprising: accessing a set of inspection images ofa set of assembly units, of a particular assembly type, recorded by anoptical inspection station during production of the set of assemblyunits; for each inspection image in the set of inspection images:detecting a set of features in the inspection image; and generating avector, in a set of vectors, representing the set of features in amulti-dimensional feature space; grouping neighboring vectors, in theset of vectors, in the multi-dimensional feature space into a set ofvector groups; labeling a first vector group, in the set of vectorgroups, with a defect indicated in inspection results of a first subsetof assembly units, in the set of assembly units, associated with vectorsin the first vector group; labeling a second vector group, in the set ofvector groups, with absence of the defect based on inspection results ofa second subset of assembly units, in the set of assembly units,associated with vectors in the second vector group; identifying a modelset of feature ranges represented in vectors in the first vector groupand distinct from vectors in the second vector group; and associatingthe model set of features with the defect.
 13. The method of claim 12,further comprising: identifying a third vector, associated with a thirdassembly unit in the set of assembly units, located between the firstvector group and the second vector group in the multi-dimensionalfeature space; serving a prompt to a user portal to inspect the thirdassembly unit for the defect; and in response to receipt of confirmationof the particular define in the third assembly unit: updating the firstvector group to encompass the third vector; identifying a revised set offeatures common to vectors in the first vector group and excluded fromvectors in the second vector group; and associating the model set offeatures with the defect.
 14. The method of claim 12: whereinidentifying the model set of features comprises identifying a firstfeature and a second feature common to vectors in the first vector groupand excluded from vectors in the second vector group; and furthercomprising: identifying a third vector representing a third assemblyunit, containing the first feature, and excluding the second feature;generating a prompt to inspect the third assembly unit for the defect;and in response to confirmation of absence of the defect from the thirdassembly unit, disassociating the first feature from the defect.
 15. Themethod of claim 12: further comprising: serving a first inspection imageof a first assembly unit to a user portal; and receiving manualselection of a region of interest in the first inspection image via theuser portal, the region of interest in the first inspection imagepredicted by a user to depict a source of the defect in the firstassembly unit; wherein detecting sets of features in the set ofinspection images comprises, for each inspection image in the set ofinspection images: projecting the region of interest from the firstinspection image onto the inspection image; and extracting the set offeatures from the region of interest in the inspection image; whereingrouping vectors into vector groups comprises grouping vectors, in theset of vectors, into the first vector group based on proximity to thefirst vector in the multi-dimensional feature space; and furthercomprising: predicting presence of the defect in a third subset ofassembly units associated with vectors in the first vector group, thethird subset of assembly units contained in the set of assembly unitsand distinct from the first subset of assembly units; and generating aprompt to inspect the third set of assembly units for the defect. 16.The method of claim 12, further comprising: selecting a first vectorrepresentative of the first vector group and associated with a firstassembly unit in the set of assembly units; selecting a second vectorrepresentative of the second vector group and associated with a secondassembly unit in the set of assembly units; detecting a common referencefeature in the first inspection image and the second inspection image;and within a user portal executing on a computing device, togglingbetween rendering the first inspection image and rendering the secondinspection image, the first inspection image and the second inspectionimage virtually aligned by the common reference feature within the userportal.
 17. The method of claim 16, wherein toggling between renderingthe first inspection image and rendering the second inspection imagecomprises animating a transition from rendering the first inspectionimage to rendering the second inspection image, virtually aligned to thefirst inspection image in rotation and orientation by the commonreference feature, within the user portal.
 18. The method of claim 12:wherein labeling the first vector group comprises: selecting a firstvector representative of first vector group; serving a first prompt to auser, via a user portal, to inspect a first assembly unit, in the set ofassembly units, for a defect, the first assembly unit associated withthe first vector; and labeling the first vector group with the defect inresponse to receipt of the first inspection result indicating the defectin the first assembly unit; and wherein labeling the second vector groupcomprises: selecting a second vector representative of second vectorgroup; serving a second prompt to the user, via the user portal, toinspect a second assembly unit, in the set of assembly units, for adefect, the second assembly unit associated with the second vector; andlabeling the second vector group with absence of the defect in responseto receipt of the second inspection result excluding an indication ofthe defect in the second assembly unit.
 19. The method of claim 12:wherein accessing the set of inspection images comprises accessing theset of inspection images comprising digital color photographic imagesrecorded by the optical inspection station over a first period of time;further comprising: receiving a second inspection image of a secondassembly unit recorded by the optical inspection station at a secondtime succeeding the first period of time; detecting a second set offeatures in the second inspection image; and in response to the secondset of features approximating the model set of features, serving aprompt to inspect the second assembly unit to a user portal atapproximately the second time, the user portal executing on a computingdevice proximal the optical inspection station.